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Abstract — Two types of noise-free relay cascades are investi- 
gated. Networks where a source communicates with a distant 
receiver via a cascade of half-duplex constrained relays, and 
networks where not only the source but also a single relay node 
intends to transmit information to the same destination. We 
introduce two relay channel models, capturing the half-duplex 
constraint, and within the framework of these models capacity is 
determined for the first network type. It turns out that capacity 
is significantly higher than the rates which are achievable with 
a straightforward time-sharing approach. A capacity achieving 
coding strategy is presented based on allocating the transmit 
and receive time slots of a node in dependence of the node's 
previously received data. For the networks of the second type, 
an upper bound to the rate region is derived from the cut-set 
bound. Further, achievability of the cut-set bound in the single 
relay case is shown given that the source rate exceeds a certain 
minimum value. 

I. Introduction 

The focus of this paper is on half-duplex constrained 
relay line networks, i. e. on multi-hopping networks where 
the intermediate relay nodes are arranged in a cascade and, 
further, are not able to transmit and receive simultaneously. 
We consider networks with a single source-destination pair and 
networks where in addition to the source a single relay node 
intends to transmit own information. Since the main interest 
is to gain a better understanding of half-duplex constrained 
transmission, we assume noiseless network links in order to 
avoid detraction from the actual topic. 

The classical relay channel goes back to van der Meulen [1]. 
Further significant results concerning capacity and coding 
schemes were obtained in [2]. More recently, the focus of 
attention shifted towards relay networks and an achievable 
rate formula for relay line networks with a single source- 
destination pair together with a random coding scheme ap- 
peared in [3]. A comprehensive literature survey as well as 
a classification of random coding strategies is given in [4]. 
There has also been work on determining the capacity or rate 
region of various half-duplex constrained relay channels [5], 
[6] and networks [7], however, under the assumption that the 
time-division schedule is determined a priori. 

An obvious approach in order to handle the half-duplex 
constraint in a line network is to use a transmission scheme 
in which even numbered relays send in say even numbered 
time slots and receive during odd numbered time slots while 



odd numbered relays behave vice versa. If the source uses 
a binary alphabet, the rate becomes 0.5 bits per use while a 
ternary alphabet yields a rate of 0.5 log 2 3 bits per use. By 
allowing randomly allocated transmit and receive time slots, 
higher rates are possible as was pointed out in [8]. In [9], 
the same author uses an entirely binary, noiseless model for 
the single relay channel such that the half-duplex constraint is 
included. It is shown there that capacity is equal to 0.7729 bits 
per use what demonstrates that time-sharing falls considerably 
short of the theoretical achievability. By the way, the same 
channel model was used in [10] in a different context. Two 
coding schemes for this particular model were outlined therein, 
which, in hindsight, can be interpreted as half-duplex schemes. 

We will introduce two further channel models for half- 
duplex constrained relays. Within the framework of these mod- 
els, it is shown that the capacity of a half-duplex constrained 
single relay channel is equal to 0.8295 bits per use if the relay 
is able to distinguish binary symbols and 1.1389 bits per use if, 
in addition, the relay is capable of detecting time slots without 
transmission. Furthermore, it is shown that the capacity of each 
relay cascade with finite length is greater than one bit per use 
assumed the latter relay model is utilized. The key idea of the 
achievable scheme is to determine the slot allocation of each 
relay node in dependence of the data received by the relay 
before. With regard to half-duplex constrained line networks, 
where not only the source but also a single relay node intends 
to transmit own information to the same destination, an upper 
bound to the rate region is derived. We finally show that in the 
special case of a single relay channel (with source and relay 
source), a slightly different version of the introduced coding 
scheme is able to achieve a segment on the cut-set bound, 
provided that the source rate exceeds a certain minimum value. 

Notation: \S\ denotes the cardinality of set S and T{S) 
the power set of S. Further, Sj := S\{i} while {f(i) : 
1 < i < to} means {/ (1), /(2), . . . , /(to)}. The conditional 
pmf Py\x(Ui x ) is indicated as p(y\x) whenever the random 
variables can be figured out from the arguments. Further, the 
vector X[ 0:m ] := (xo, x%, . . . , x m ) summarizes realizations of 
the random variables Xq, X\, . . . , X m . The entropy expression 
H(Yi\X( k:k>1 )) equals H(Yi\X k ) in case k > 1 and H(Yi) 
in case k < 1. We will abbreviate pxtX i+1 (a, b) as p l ab . 

II. Network Model 
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Fig. 1 . The considered multiple relay cascade (top) and an excerpt. If relay i 
is transmitting, the switch is in position 1 otherwise in position 2. 

number from the integer set {0, . . . , to + 1}. The integers 
and m+1 are allocated to source and destination, respectively. 
The remaining nodes 1 to m represent half-duplex constrained 
relays (abbreviated as HD relays). A graphical representation 
is given in Fig. [TJ The output of the ith node, which is the 
input to the channel between node i and i + 1, is denoted 
as Xi, i G {0, ...,m), and takes values on the alphabet 
Xi = {0, 1, N}, where N is meant to signify a channel use in 
which node i is not transmitting. Correspondingly, the input 
of the ith node, which is the output of the channel between 
node i—1 and node i, is Yi, i G {1, . . . , m + 1}, with values 
from the alphabet y^. Each message Wq, sent via multiple hops 
from source node to sink node m + 1, is uniformly drawn 
from the index set Wo = {1, . . . , 2 nR °}, where n is the block 
length of the encoding scheme and Rq the transmission rate. 
Apart from the source node, there is possibly a single relay 
node r G {1, • • • , to}, which intends to transmit independent 
indices taken from W r — {1, • ■ • , 2 nRr } to the destination. 
Again, the transmission scheme is multi-hopping since the 
information flow associated with message w r has to pass all 
nodes with indices greater than r. We assume noiseless links 
what results in a deterministic network, i. e. the entries in 
p(y\i:m+l] l x [0:ml) are either or 1. In order to introduce the 
half-duplex constraint, we impose following channel model 
onto each relay node i G {1, . . . , m} 

Xi-x, ifX,=N 
X u if Xi G {0, 1}, 

where Y m+ i = X m . Relay model (fl} is denoted as ternary 
since the reception alphabet of each relay node is 3^ = 
{0, 1, N}. It can easily be verified that y t = {0,1} when (N,N) 
is excluded from the Cartesian product x Xi, and in this 
case the model is referred to as binary. The interpretation of 
both models is as follows: in case relay i sends binary data, 
i. e. Xi G {0,1}, it only hears itself and, thus, cannot listen 
to relay i — 1 or, equivalently, relay i and relay i — 1 are 
disconnected. Conversely, if relay i is quiet, i. e. Xi = N, it 
is sensitive for the channel input of relay i—1. The feedback 
interpretation of the relay nodes as shown in Fig. [TJ results 
from these considerations. As a consequence of the underlying 
model, the conditional channel pmf can be factored as 

P (yp.:m+l] |X[ 0:m ]) = p(yi\X[0:l])-'-p(y m \X[m-l:m]) 

p(y m+ i\x m ). (2) 

Moreover, we will assume that the channel inputs 
Xq , X\ , . . . , X m form a Markov chain what seems to 
be unmotivated at first glance but turns out to be without loss 
of optimality as explained in Remark [3] 
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Theorem 1: The zero-error capacity of the relay network 
defined above, where only the source but no relay transmits 
own information, is given by 



C = max vaw.{H(y x \Xx), 

P( X [0: m ]) 



,H(Y m \X m ),H(Y m+1 )}. 



(3) 

Proof: The proof is given in the Appendix. Achievability 
is shown in the next section. ■ 
Example 1 (Single HD Relay Channel, m = 1): The con- 
sidered channel with a ternary relay falls into the class of 
degraded relay channels [2]. At each time instance, the relay 
is either listening or transmitting. When the relay transmits, 
i. e. xi G {0, 1}, the source input cannot be detected by the 
relay and, consequently, the source should not transmit. Thus, 
it can be assumed w.l.o.g. that Pq = p^ = p1 Q = p^i = 0. 
Hence, the source input is not random when x\ G {0, 1} and 
together with (|T), equation (0) reduces to 

C = max min {H(X \X X = N)p Xl (N), ff(Xi)} . (4) 

P( X [0:1]) 

However, when the relay is listening, i. e. x\ = N, the source 
should make optimum use of the channel by encoding with 
uniformly distributed input symbols, i. e. p® m = pJ N = p^ N . 
Furthermore, in order to achieve the maximum information 
flow H(Xi) from the relay to the sink or, likewise, from 
a symmetry argument, we can choose p^ — Pni- These 
considerations yield a single degree of freedom in (0|. Since 
the maximum does not occur in the maximum of one of the 
two concave functions, (01 is solved by H(Xq\X x ) = H(Xi). 
The resulting assignment is p[| N = p\ n = p^ N = 0.2395 and 



Pno 



P°m 



= 0.1407, which yields C = 1.1389 bits per 
channel use. 

Remark 1: Evaluation of capacity for the binary HD model 
is almost along the same lines as in Example [TJ However, the 
channel input x xi = NN is not allowed in the binary model 
and, thus, we a priori have p^ N = 0, which yields C — 0.8295 
bits per channel use. 

Example 2 (Infinite HD Relay Channel, m — ► oo): All re- 
lays in the cascade behave according to the ternary model. Due 
to the Markov property of the channel inputs, the joint pmf 
p(x[o :m ]) is completely characterized by p(xrrj : i]), p(x[ 1:2 ]), 
.... P(x [m _i :m] ). Further, H(Yi\Xi) = HiX^X,), which 
follows from ([TJ. The idea is now to find a probability 
assignment such that the p(xu_ 1: i]) are equal for all i G 
{1,2,..., to} without violating any optimality requirements. 
If we can find such a probability assignment, capacity simply 
follows by maximizing a single H(Xi-i\Xi) for that partic- 
ular assignment. We now pick an arbitrary positive integer i 
and try to make p 1 ^ 1 and p l kl equal for all combinations k, I G 
{0,1, N}. By the same arguments as in Example [TJ we can 
choose w.l.o.g. p'qq 1 = p 1 ^ 1 = p 1 ^ 1 = p 1 ^ 1 = 0, and the same 
is valid for p(x[ i: j +1 ]). As a simple consequence, p 1 ^^ — Pq N 
and p^vi = Pin anc ^ from a symmetry argument, p 1 ^ = p 1 ^ 
and p l 0N = p\ N - Further regarding our objective, we have to 
require that p 1 ^ = p l kN for k G {0, 1}. Since index i has 



been picked arbitrarily at the beginning, the procedure is valid 
for each p(x«_i 3 i) and p(x[ i:i+1 ]), 1 < i < m — 1, what 
is sufficient in order to achieve equal pmfs with a common, 
single degree of freedom (e. g. p n)- Hence, 
1 < i < m, is easy to optimize yielding H(Xi-i\Xi) = 1 bit 
achieved at p 0N = g. The capacity C is, therefore, equal to 
1 bit per channel use. 

Remark 2: Application of the binary HD relay model yields 
C = 0.5 bits per channel use for all relay cascades composed 
of two or more binary HD relays. Therefore, the optimum 
transmission strategy is just a straightforward time-sharing 
approach. The reason lies simply in the fact that the relays 
cannot encode parts of their information by means of the slot 
allocation since the subsequent relay is not able to recognize 
when nothing (i. e. symbol N) was sent. 

Theorem 2: The rate region of the relay network defined 
above with two sources, namely source node and relay 
node r, is characterized by 

R < max min {H (Yi\Xi) : 1 < i < m} (5) 

P(*[0:m]) 

R r < max mm{H (Yi\X r -i,Xt i:i < m )) : 

P( X [0:n,]) 

r + l<i<m + l} (6) 

and (|7|) shown at the bottom of the page. The maximization 
of the equations is performed jointly regarding p(xr 0:rn ]). 

Proof: The proof is given in the Appendix. ■ 
Example 3 (HD Single Relay Network with Two Sources): 
The ternary relay network considered here is characterized by 
m = 1 and r = 1 and together with (Q]), Theorem [2] becomes 

R < H(X \X!) (8) 
R 1 < HiX^Xo) (9) 
Ro + Rx < H{Xx). (10) 

An outer bound on the rate region of the considered line 
network is obviously given by Rq + Ri < log 2 3 bits (Fig. [2] 
graph (a)) since the sum-rate can never be larger than the 
maximum of H(Xi). We first try to determine whether points 
on this outer bound, besides (Rq,Ri) = (0,log 2 3) bits, are 
delivered by equations (8) to (10) what inevitably requires 
a uniform px 1 {x\). Since H(Xq\Xi) has to be smaller or 
equal to H(Xi), we are allowed to assume equality in (8) 
what follows from TheoremQ] By making the same optimality 
assumptions regarding p(xro : ii) as in Example Q] we get 
i?o = ^log 2 3 bits and, consequently, Ri < |log 2 3 bits. 
Note that this value for Ri does not contradict with (9), i. e. 
it is smaller than H(Xi\X ) concerning the assumed input 
distribution. The obtained point lies on the outer bound and 
it follows from a time-sharing argument that all points on the 
line between (0, log 2 3) bits and (ilog 2 3, |log 2 3) bits are part 
of the rate region bound characterized by (8) to (10). 



In the sequel, we maintain the optimality assumptions re- 
garding p(x[Q;i]) and focus on the remaining interval ^log 2 3 < 
Rq < 1.1389 bits, where 1.1389 bits is the capacity of a 
single ternary HD relay channel (Example [TJ. Again, Rq = 
H(Xo\X\) but now px 1 {xi) is not uniform anymore (due to 
Ro > -|log 2 3) yielding a sum-rate strictly smaller than log 2 3 
bits. An upper bound on R\ is given by H(Xi) — Rq. It 
remains to check whether this expression is smaller or equal 
to the right hand side of (9) in the considered interval for Ro 
for the assumed input distribution. However, this is satisfied 
and, therefore, the complete upper bound on the rate region 
according to (8)-(10) is characterized by 

J log 2 3 -Ro, 0<Ro< |log 2 3 

Rl -\H b (sgg) + (l - jgs) - Ro, ilog 2 3 < Ro < C, 

where Hb(-) denotes the binary entropy function and C = 
1.1389 bits per channel use. A graphical representation is given 
in Fig. |2] graph (b). 

IV. Coding Strategies 

A. Achievability of C in Theorem Q] 

A coding strategy is presented capable of achieving C in 
Theorem Q] As it is standard in achievability proofs, blocks of 
transmissions are used such that in B blocks a sequence of B— 
m indices wq € Wo is sent from the source to the destination. 
As B -> oo, the rate flo( ^ m) -> Rq. The idea behind the 
coding strategy is the following. Based on the feedforward 
property of the considered line network and due to the fact 
that each node is aware of the encoding strategy used by nodes 
with larger indices, node i, < i < m, knows at each time 
instance the codeword, which will be sent by nodes I > i in 
the upcoming transmission block. Thus, each node is able to 
adapt its transmission to the codeword chosen by the next node 
what can be exploited in order to prevent that concurrently sent 
codewords of adjacent nodes occupy the same time slots with 
binary symbols {0,1}. 

Different techniques for encoding are used by the source and 
the ternary relays. While the source utilizes a ternary alphabet 
{0, 1,N} for encoding, the relays represent their messages by 
a combination of binary symbols {0, 1} and the allocation of 
binary symbols to the slots of a transmission block. Let rij 
denote the number of binary symbols used by relay i during 
a single transmission block. Then, at most 2™ m (J 1 ) indices 
can be encoded by relay m where 2™ m denotes the number 
of distinctive indices when the binary symbols are located at 
fixed slots while ( n ) denotes the number of possible slot 
allocations. Due to the half-duplex constraint, the effective 
codeword length of relay to — 1 reduces to n—n m . This results 
from the fact that relay to cannot pay attention to relay to — 1 
when relay to sends binary symbols and, therefore, the number 
of indices, encodable by relay to— 1, is at most 2 rim - 1 (™~™ m ) ■ 



Ro + Rr < max min{min{ J f/(y i |X i ) : 1 < i < r - 1} + mm{H(Y k \X, T ^ 1:r -i> 1) ,Xr h:h < m) ) : r + 1 < k < m + 1}, H(Y m+1 )\ 

P( x [0:m]) 

(7) 



The same argumentation holds for each relay in the chain, i. e. 
relay i, 1 < i < to, is able to encode at most 2 ni ( n_ " i+1 ) 
indices per transmission block where n m+ i = since the sink 
node listens all the time. Finally, the effective length of the 
source codeword is n~ni what enables the source to encode 
a maximum of 3 n ~ ni indices. The rate R — n~ 1 log 2 |Wo| is 



Rq 



log 2 3, — 

n n 



1 



;log 2 



n - n l+1 



(11) 

where 1 < i < to. 

Codebook Construction: The source and all relays generate 
codewords according to the scheme described in the previous 
paragraph. Let Wi G Wo indicate a message index forwarded 
by relay i, and let Si G Si denote a particular slot allocation 
used by relay i for encoding indices w;. Note that each 
consists of n — rii + i slots, which can be embedded in at most 
(n™ J wavs mto a block of length n whereas the embedding is 
a function of the concurrently used s^+i ,s m . The resulting 
slot allocations of length n, employed by relay i, are denoted 
as Zi G Zi and depend on Sj, . . . , s m . The procedure works 
as follows. Fix |Wo| relay to codewords x^^Wm). For each 
slot allocation z m used in relay to codewords, construct |Wb| 
relay m — 1 codewords x^„ 1 (w m _i, z m ). This ensures that 
relay to — 1 can encode each message w m -\ independently of 
the slot allocation used by relay m. The procedure repeats and, 
finally, for each slot allocation z\ used in relay 1 codewords, 
construct |Wo source codewords x^(wq,z\). 

Encoding {at the end of block b — 1): Let Wq G Wo 
denote the new message chosen by the source to be sent in 
block b, and let G Wo denote the estimate of w^ 1 ^ 
made by relay i at the end of block 6 — 1. Further, s[ , 



which is a function of w 



(b) 



corresponds to the slot allocation 

,( b ) 



used by relay i in transmission block b for encoding w 4 
whereas zf^ is determined by s\ b ' , . . . , §m ■ Relay node m 

•(b), 



sends x^{wm ) in block b. Since relay node i, 1 < i < m — 1, 
knows all previously sent indices (w 



f(b-l) ~(b-2) 



equal (wS^, t&|2 2 ' • • •)> ^ knows and encodes its latest 
index u) t - with x" (w^ , z^. 1 ) . Similarly, the source chooses 
Xq (w^ , Zi) for transmission in block b. 

Decoding (at the end of block b—1): At the end of block b— 
2, relay i has estimates (u> t - 1 ',w) l - b 2 \...) and, therefore, 
i ^i+i 1 ^ • • • ) an( l °f z^ 1 . Then, based on 



CO 



which 



estimates of (s; . .s 



the received sequence xf_ 1 (wf^-, 1 ^ , zf 1 ^) during block 6—1 
and due to the knowledge of the codebook used by relay i — 
1, relay i is able to determine the unknown index w^_x • 
The destination knows the codebook used by relay m and 
upon receiving x^vbm it can determine Wm ^ ■ Both 
the codebook construction and the noise freedom of the relay 
cascade guarantee, that the decoding steps can be performed 
with zero-error probability. 

Achiev ability: Using the relation n~ 1 log(^ i ) = iJf, (^) 
[11, Th. 1.4.5] as n — > oo, optimality assumptions regarding 
p(x[j.j +1 j) (symmetry, zero probabilities - see Example[2]), the 
resultant identities ^ = p 0N +p* 1N and = Pnn' we 
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Fig. 2. A single ternary HD relay channel with two sources is considered, 
(a) Bound due to single source capacities, (b) Upper bound due to Theorem|2] 
(c) Region due to the coding strategy with block length n = 640. 

obtain 



m 1, fn-rii+i 

— + -log 2 

n n \ m 



where 1 < i < to. According to the model in (Q~|), 
H(Xi\Xi + i) — H(Yi + i\Xi + i) what shows that each entry 
in (fTTb . except for the first, converges to the corresponding 
entry in (01. The first entry in ( fTTT i corresponds to a source, 
which uses uniformly distributed input symbols when relay 1 
is listening. Evaluation of H(Yi\X±) regarding a uniform 
Px \Xx ( x o, N) yields pxi(N)log 2 3. Hence, the first entry in 
OJ equals the first entry in (TTTT) . ■ 
Remark 3: At this point, we are able to justify why it has 
been without loss of optimality to impose the Markov property 
on the channel inputs. Assume that each pair of channel inputs 
is statistically dependent given all remaining inputs. Then the 
procedure regarding Theorem Q] as shown in the Appendix, 
yields maxmm{H (Yi\X[i :m ]), H(Y m +i) : 1 < i < to} as 
simplified cut-set bound what is smaller or at most equal to the 
achievable rate. But since the cut-set bound is an outer bound, 
only equality is valid, achieved e. g. by Xq X m . 
For non-Markovian inputs, the rate region bound as stated in 
Theorem |2] is still an upper bound (but eventually looser). The 
Markov property merely cancels conditional random variables 
from the entropies what does not reduce the region. 

B. Coding Strategy for a HD Relay Cascade with Two Sources 

A coding scheme based on similar ideas can be derived for 
a line network where a second relay node r intends to transmit 
own information. Two main points have to be considered: 

> Relay source r and all subsequent relay nodes must be 
able to encode |Wo| ■ |W r | different indices since Wq and 
W r are independent. 

> The slot allocations z r G Z r , applied by relay source r, 
are completely determined by the source indices wq. 

Theorem 3: Consider a ternary single HD relay channel 
where both source and relay send own information. The bound, 
described by equations (8) to (10), is achievable provided that 
the source rate exceeds a threshold. 

Proof: Let tn\ and (1 — t)m denote the number of 
binary symbols used by the relay for encoding each wq and 



Wx, respectively, where < t < 1. Further, all possible slot 
allocations of the relay represent indices wo. If the number 
of source indices matches the number of relay codewords for 
representing source indices, or expressed in i?o 

n — ?i i tn-\ 1 / ii \ 

-log 2 3 = — + -log 2 , 0<t<l, (12) 

n n n \nij 

the cut-set bound is achievable. Note that the lhs of ( fT2l 
equals pxi(N)log 2 3 what in turn equals H(Xq\Xi), assumed 
the same p(x[ 0: i]) is used than in Example Q] Further, R\ = 
(1 — tjnin^ 1 . As n — ► oo, Rq + R\ — ► H(X\) what results 
from [11, Th. 1.4.5] under consideration of the rhs of (fT2l) . 
The minimum Rq (threshold) follows from dT2b for t — 0. ■ 

V. Appendix 

Proof of Theorem Q} An upper bound on the capacity of 
each single source-destination network with source and sink 
node to + 1 is given by [12, Th. 14.10.1] 

C< max mmI(X ,X s .;Y s ,Y m+1 \X s ), (13) 

P(x[0:m]) SEM 

where M. — V({1, ■ ■ ■ ,to}) and S c is the complement of S 
in {1, . . . , to}. In case of a noise-free network, ( fT3l becomes 

C< max mmH(Y s ,Y m+1 \X s ). (14) 

P(x [0:m ]) S6 - M 

Let S be nonempty and let Z € {1, . . . , to} denote the smallest 
integer in S. Then 

H(Y s ,Y m+1 \X s ) > i/(y ; |^s f ,^) + ^(>Srl^ f ,^,^) 
( = } 11041X1) + H(Xs T \Xs v X u Yi) 

> H(Yi\Xi), (15) 

where (a) follows from the chain rule and (b) from Xg r — > 
X; — > Y;. Equality in (a) and (c) is achieved by the 
ascending index sets S — {1,1 + 1, ...,m}, 1 < I < to, 
which compose the entries of a set say M. a . Hence, for 
each S' e M\{%} there exists an S 6 .M a such that 
H(Y S , Y m+1 \X s ) < H(Y s ,,Y m+1 \X s >). Take e. g. S' = 
{1,1 + v}, where < v < to — I, and extend it to an ascending 
index set S = {1,1 + l,...,m}. The claim, stated in the 
sentence before the last, holds. In summary, (fTBT l yields the 
first m entries in (O whereas the remaining entry, H (Y m+ i), 
follows when S in (TT4T > is replaced by the empty set. ■ 
Proof of Theorem [2} The derivation of the individual 
rate bounds is almost along the same lines as in the proof of 
Theorem Q] Hence, we concentrate on the sum-rate bound. 

An upper bound on the sum-rate of each network with two 
sources and r and a sink to + 1 is [12, Th. 14.10.1] 

R + R r < max min I{X , X r , X s <=; Y s , Y m+1 \X S ), 
p(x [0:m i) seM 

(16) 

where M is the power set of M d U M u := {1, . . . , r - 1} U 
{r + 1, . . . ,m}. Note that the rhs of (fToT l simplifies to the rhs 
of ( TBI due to the assumed noise freedom. Let S d s V(M ) 
and S u S V{M U ) where S = S d U S". First let S d and S 111 



be nonempty, i. e. M! := V(S) C A4. Further, let i and j be 
the minimum and maximum values in S d whereas k denotes 
the minimum value in S u . Then 

H(Y s ,Y m+1 \X s ) > H(Y t ,Y k \X s ) 

+H(Y sd ,Y s ^\X s ,Y l ,Y k ) 

i fc 

= HiYi, Yk\Xi, Xj , Xk) 

+H{Y st ,Y sl \X s ,Y l ,Y k ) 

> H(Yi\Xi) + H(Y k \X r ^,X k ), (17) 

where (a) follows from the chain rule, (b) from 
{X s4 ,X S u) ^X t ^Y % and (X s| ,X sr ) -> (X„ X fc ) -» F fc , 
and (c) from applying chain rule to the first term in (b) under 
consideration of (Xj,X k ) — > (X r _i,ATfc) — > Yfc together 
with the described Markov relations. Equality in (a) and (c) 
is achieved by the ascending sets S d = {i, i + 1, . . . , r — 1}, 
1 < i < r - 1, and S u = {k, k + 1, . . . , to}, r + 1 < k < to, 
which compose the entries S — S d U S u of a set say 
.M Q . Then for each S' £ M' there exists a S £ M a such 
that H(Y s ,Y m+1 \X s ) < H{Y s ,,Y m+1 \X s >). Take e. g. 
5' = {i, i + v} U {fc, /c + w}, where 0<w<r — 1 — i and 
< w < m — k, and extend S 1 to an ascending index set 
S = {i, i + 1, . . . , r — 1} U {k, ft + 1, ... , to}. The inequality 
relation holds. In summary, the procedure yields 

minjiiXYilXi) : 1 < i < r - 1} 

+ min{H(Y k \X r _ 1 ,X k ) : r + 1 < ft < to} 

in ©, what follows from (TTTb taking into account all com- 
binations of i and k. The last entry in ^ and the modified 
version of above equation in (|7]i result when, in addition, the 
sets S € M\M' are considered (S d , S u empty or both). ■ 
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